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1. Introduction 

Statistical models for phenomena that evolve over time typically assume that 
observations arise from the discrete sampling an underlying, continuous-time 
stochastic process. In this setup, asymptotic results are often obtained as the 
sampling rate goes to infinity, i.e., as (5 — > 0, where 5 is the (average) time 
between two consecutive sampling instances. Such a high-frequency setup has 
been used, for example, in order to estimate the quadratic variation of an Ito 
process [31]; test for the presence of jumps [1] and the degree of their activity 
[2] in a semimartingale; estimate the Hurst index of fractional Brownian motion 
[9] and other self-similar processes [27]. 

High-frequency sampling is a natural assumption in financial modeling (see, 
e.g., [13]), where tick by tick price signals are widely available. However, in other 
applications areas, there is, or it may be desirable to have, infrequent sampling. 
This is, for example, the case when the process of interest is being monitored at 
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some remote location by a network of low-cost, battery-operated sensors, which 
transmit summaries of their data to the statistician [14]. In this context, infre- 
quent communication leads to significant energy gains, since it allows shutting 
down the sensors for long time-periods, during which they only sense the envi- 
ronment and do not communicate their values [20]. Such applications have led 
to the modification of existing statistical procedures and the reformulation of 
classical statistical problems so that communication constraints be taken into 
account (see, e.g., [21], [4]). 

Motivated by such applications, our goal in the present work is twofold. First, 
to extend some fundamental asymptotic results from renewal theory in a low-rate 
setup, i.e., as the rate of the renewal process goes to 0. Second, to argue, using 
both asymptotic theory and simulations, for the necessity of low-rate sampling 
in the prototypical statistical problem, that is the estimation of the drift of a 
random walk, when the random walk is sampled at a sequence of random times 
so that its increments are censored. 

Censoring arises naturally in applications that involve wireless sensor net- 
works. Indeed, such networks are typically characterized by limited bandwidth, 
which implies that the sensors need to transmit only low-bit messages (see, e.g., 
[6], [29]). In order to achieve this, it is typically suggested that the sensor ob- 
servations be quantized, using some thresholding scheme [28]. An alternative 
strategy, however, is to communicate whenever the statistic of interest increases 
(or decreases) by a certain amount in comparison to its value at the previous 
communication time. In this context, censoring means ignoring the excess, or 
overshoot, of the statistic above (or below) its target level at each communica- 
tion time and, consequently, allows the transmission of only one-bit messages 
from the sensors. 

This kind of random sampling has been applied in different statistical prob- 
lems in the context of sensor networks, such as linear state estimation [19], 
sequential hypothesis testing [10], sequential change detection [11], (sequential) 
parameter estimation [12]. In the latter work, it was assumed, for the most 
part, that the sensors observe continuously the paths of, possibly correlated, 
continuous-path semimartingales, an assumption which precluded loss of infor- 
mation due to unobserved overshoots. Discrete sampling (and the associated 
overshoot effect) was considered in the case that the sensors observe indepen- 
dent Brownian motions with common drift ^ at discrete, equidistant times with 
common period h. In this context, it was shown that ignoring the overshoots 
does not imply any asymptotic loss of efficiency with respect to the estimation 
of ^, as long as the underlying Brownian motions are sampled at a sufficiently 
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high rate [h 0). 

In the present work, we give a much more complete answer to the more gen- 
eral problem of estimating the drift fj. of an arbitrary random walk {St)t£H with 
variance a^, i.e., ^[St] = and Var[S't] = a^t. That is, the random walk is not 
necessarily Gaussian and does not necessarily arise from the discrete sampling 
of an underlying continuous-time process. (We consider a scalar process only in 
order to lighten the notation, all statements below can be generalized immedi- 
ately to an arbitrary number of independent random walks with common drift). 
From the Central Limit Theorem we know that the sample mean, /if = St/t, is 
an asymptotically normal estimator of /i, in particular, 

%/t(/if - /i) ^7V(0,cr^) as i-^oo. (1.1) 

Suppose now that the random walk is only observed at a sequence of random 
times (T„)„gN that form a renewal process with rate 1/5, i.e., (t„ — T„„i)„gN 
are integer-valued, i.i.d. random variables with the same distribution as t := ti 
and mean 5 := E[t]. For any fixed sampling period 5, it is easy to see that (1.1) 
remains valid when we replace either by fit '■= Srf^^/TNt or by jit := S'r„^/t, 
where Nt = max{n G N : t„ < t} is the number of acquired samples up to time 
t. 

In Theorem 3.1, we show that this is also the case when the sampling rate 
is asymptotically low, i.e., when we not only let t — >■ oo, but also (5 — ?> oo. More 
specifically, we show that if the variance of the time between two consecutive 
sampling instances is at most a quadratic function of the average sampling 
period (more precisely, if Var[T] = 0{5^) as 5 — oo), then (1.1) remains valid 
as (5, t — > oo when we replace by fit (resp. /it), as long as (5 = o{t) (resp. 
6 = o{y/t)). Thus, whereas the two estimators are indistinguishable in terms of 
their asymptotic distributions as t — ?> oo when 5 is fixed, letting 5 oo reveals 
that ftt is more robust than fit to large sampling periods. 

However, the main emphasis of the present paper is on the case that the 
increments of the random walk are censored, in the sense that {St) is sampled 
whenever it increases by a known amount A > since the previous sampling 
instance, i.e., 

r„ = inf{< > r„_i : St ~ Sr„_, > A}, n e N, (1.2) 

but the overshoots {Sr„ — S't„_i — A)„gN are unobserved. In other words, at 
any sampling time t„, we do not learn by how much more than A has the 
random walk exceeded its value at t„_i. In this context, our goal is to con- 
struct estimators that rely only on the counting process, (iVt), and yet preserve 
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the asymptotic distribution (1.1). It turns out that this is possible only under 
appropriate, asymptotically low sampling rates. 

More specifically, in Theorem 3.2 we show that (1.1) remains valid when we 
ignore the overshoots in jit and estimate n with (it ■= {NtA)/TNt, as long as 
y/t « S « t. On the other hand, ignoring the overshoots in jit and estimating 

with jit := {NtA)/t fails to preserve (1.1) at any sampling rate. Nevertheless, 
when the distribution of the random walk is known and satisfies certain condi- 
tions, we suggest a correction for the overshoots which leads to modifications of 
fit and jit that both preserve (1.1) when << S « t and \/t « S « Vt, re- 
spectively (Theorem 3.3). For this overshoot correction to work, wc require that 
the average overshoot converge sufficiently fast to its limiting value as A — > oo 
and that the latter be a Lipschitz function of fi; the second requirement, which 
is more restrictive, is satisfied for various distributions when the standard devia- 
tion a is proportional to /i. Moreover, we support these findings with simulation 
experiments, with which we compare the above estimators in terms of their 
mean square errors in the context of a curved Gaussian distribution. 

In order to establish these asymptotic results, we need to extend the classical 
renewal theorem, according to which the empirical sampling rate Nt jt converges 
in to the theoretical rate 1/^asi— )-oo, ina low-rate setup, that is as the 
sampling period 5 goes to infinity. We obtain such an extension, not only for 
the renewal process defined by (1.2), but for arbitrary renewal processes and, 
more generally, for random walks. 

Indeed, it is well known (see Theorem 8.1 in [16]) that if (t„) is a random 
walk with positive drift E and E[(t~)'"] < co for some r > 1, then for any fixed 
5 > 0, 

Vt 1 

> - as i — > oo, 

t 

where vt '■— inf{n G N : r„ > t}. Moreover, in the special case of renewal 
processes, i.e., when P(r > 0) = 1, wc can replace i^t by Nt, since lyt ~ Nt + 1, 
In Theorem 2.2, we show that this asymptotic result remains valid as (5 — > oo, 
in the sense that 

Sly 

— >^'^ 1 as S,t ^ oo so that S ~ o{t), (1.3) 

as long as E[|t - Jp^^ = 0(6^'^'') and E[(t+)''+1] = 0(6''+'^). In the special case 
of renewal processes, these requirements reduce to E[r''+^] = 0((5''+^) and are 
satisfied when, for example, r is stochastically less variable than an exponential 
random variable, or when r = inf{t : Yt > A}, where (Yt) is spectrally negative 
Levy process (or a random walk) so that E[Yi] > and E[(Yj^^)''+^] < oo. 
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Moreover, although it is not needed for the estimation problem we consider 
in this work, we also obtain a low-rate extension of the following classical result 
from renewal theory (see Theorem 9.1 in [16]): 

Var[j/t] Var[r] 

> — -p^ — as t ^ oo, 

t o-^ 

which holds for any fixed 5, whenever Var[r] < oo. In Theorem 2.3, we show 
that this asymptotic result remains valid as (5 — >■ cxj, in the sense that 

j3 Varffjl 

— 1 as (5, t— >oo so that 5 = o{t), (1.4) 



VarH t 



as long as Var[T] = e((52) and E[(t+)3] = 0{5^). 

We are not aware of extensions of renewal theorems in which the rate of 
the renewal process is asymptotically low or, more generally, the drift of the 
random walk is asymptotically large. On the contrary, in statistical problems 
it is usual to consider random walk with small drifts. For example, when (t„) 
is a random walk so that the distribution of r can be embedded in a natural 
exponential family, Siegmund [23] (see also [24], Chapter 10) computed the 
limiting value of E[(ti/j — ty] as (5 — > and t — > oo so that 5t = 0(1) in order 
to obtain corrected diffusion approximations to the operating characteristics 
of the Sequential Probability Ratio Test. (For more refined corrected diffusion 
approximations, we refer to [7], [8], [5]). 

The rest of the paper is organized as follows: in Section 2, we assume that 
(t„) is a random walk and establish (1.3) and (1.4). In Section 3, we consider 
the estimation of the drift of a random walk {St) that is observed at an arbitrary 
renewal process (t„) and, in particular, when (r„) is given by (1.2) and censoring 
is present. We conclude in Section 4. 



2. Low- rate renewal theory 

Let (T„)„gN be a random walk with positive drift, i.e., let (r„ — r„_i)„gN be a 
sequence of i.i.d. random variables with mean 5 := E[r] > 0, where r :~ ri, and 
consider the first-passage time process 

vt := inf{7i G N : T„ > t}, t> 0. (2.1) 

When (t„) is a renewal process, i.e., P(t > 0) = 1, z^t = A^t + 1, where {Nt) is 
the corresponding renewal counting process, 

Nt := max{7i e N* : t„ <t}, te N. (2.2) 
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For any t > 0, t't is a stopping time with respect to the filtration generated by 
(r„), thus, 

vt 

Tut - ^'^t = '^{'Tn - Tn~l - S) 
n=l 

is a stopped random walk with drift zero. Thus, from Wald's first and second 
identity we have 

E[r,J = E[n] S, Var[r,, - 5^] = E[(r,, - Snf] = Eh]Var[r], (2.3) 

whereas from the stopped versions of the Marcinkiewicz-Zygmund inequalities 
(see [16], Theorem 5.1, page 22) it follows that for any r > 2: 

E[iru,~5,.tY]<CrEy/']E[\r~Sa (2.4) 

where by Cr we represent a generic, positive constant that depends on r. 

Moreover, from Lorden [17] we have the following bound on the expectation 
of the residual lifetime/excess/overshoot, t^^ — t: 

and, more generally, for any r > 1 we have: 

E[{ru,-tY]<Cr J Y (2.6) 
We will also use the following algebraic inequality, which holds for any r > 1, 
\x-yr <crix'' +yn, x,y>0. (2.7) 

2.1. A low-rate renewal theorem 

We first establish (1.3) when r = 1. 

Theorem 2.1. //Var[r] = 0{S^) and E[{t+Y] = 0{S'^), then 
Sly 1 

— >^ 1 as S,t ^oo so that S = o(t). 

Proof. Let us first observe that 
Svt 



- 1 
t 



<Mf^i-^ + !ili_^. (2.8) 



From (2.5) and the assumption that E[(t+)^] = 0{S'^) it follows that 

E[T.,-t] ^ E[{t+Y] OjS') (2.9) 



t ~ St St \t 
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Moreover, starting with an application of Holder's inequality we have 



H\5^t ~ r.J] ^ , im^t - r,.Y\ _ . /ENVar[r] _ /E[r,J VarH 



t - V t2 V t2 \l t St 



where the first two equalities follow from Wald's identities (2.3), whereas the 
third equality follows from (2.9) and the assumption that Var[T] = C(5^). Taking 
expectations in (2.8) and applying (2.9) and (2.10) completes the proof. □ 

Using Theorem 2.1, we will now establish (1.3) for arbitrary r > 1. In order 
to do so, we will need the following condition 

(C) E[|r - Sr^] = 0((5'-^2) ^^^d E[(r+)'-+i] = 0((5''+i), 

which reduces to the assumptions of Theorem 2.1 when r = 1. Moreover, as 
we show in the following lemma, if condition (C) holds for some r > 1, then it 
remains valid for any s G [0, r]. 

Lemma 2.1. If condition (C) holds, then 

E[(r+)^] =0(5^), V,se[0,r + 1], 
£[\t-5Y]=0{5''), V.se[0,rV2]. 

Proof. We will only prove the first claim, since the second can be shown in a 
similar way. Thus, from Holder's inequality and condition (C) it follows that for 
any s g [0, r + 1] 

E[(r+)«] < (E[(T+)'-+i])^ = (0((5'-+i))^ =0(<5«), 

which completes the proof. □ 

Finally, for any ?' > 1 we will use the following notation 

E[|r-5n=e(J^'-), a.:=2^-l, (2.11) 

where is some positive number. Condition (C) implies that grv2 < r\/ 2 and 
consequently < 1. However, we will see that the exact value of (7rv2, and in 
particular Ur, determines the rate of convergence in (1.3). 

Theorem 2.2. Let r >\. If condition (C) holds, then 
5vt 



t 



and, consequently, 
6vt 



^ '1 as 5,t^oo so that S = o(t). 
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Proof of Theorem 2.2. Due to condition (C), < 1, thus it suffices to prove 
(2.12). Appiying ttie algebraic inequaiity (2.8) to (2.7) and taking expectations 
we have 



5vt 

T 



1 



< Cr 



From (2.5) and condition (C) it foiiows that 

E[(r,,-i)1/„ E[(r+)-+i] _ 0(^-^+1) 



< Cr 



6r 



6'' 

= o{- 



(2.13) 



(2.14) 



From an appiication of Holder's inequality and (2.4) it follows that 



|rV2i 



< C, 



Cr E 



and, consequently, 

E[\Siyt - T„ 



< CrE 

= E 



<5r 



E[|t-(5|'-^2]^ 



5vt 
T 



o 



6^ 

T 



(2.15) 



where the equality follows from (2.11). Thus, from (2.13)-(2.15) wc obtain 



T 



1 



< o 



t 



O 



t 



When 1 < 7' < 2, from Theorem 2.1 wc have 









— )• 1 as 


Kfr] 


= E 






I t \ 





— )• 1 as (5, t-H'Oo so that 6 ^ o{t). 

Indeed, from Lemma 2.1 it follows that condition (C) holds for r = 1, in which 
case it reduces to the assumptions of Theorem 2.1. Therefore, (2.12) holds when 
1 < r < 2. 

When 2 < 7' < 4, it suffices to show that 



ovt\~ 



5vt \ h 
1 



1 as (5, s-oD so that S — o{t). 



Indeed, from Lemma 2.1 we know that condition (C) holds when r is replaced 
by r/2 and we just proved that the Theorem holds for r/2 G (1,2), which 
proves (2.12) for 2 < r < 4. The theorem can be shown in a similar way for any 
4 < r < 8, etc. □ 
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Remarks: 

1. When (7rv2 = f V 2, then a,, = 1 and, consequently, 



2. When qr\/2 



- 1 



= o 



then Or = and, consequently, 



SNt 



- 1 



= O 



2.2. The case of renewal processes 

When (t„) is a renewal process, i.e., P(t > 0) = 1, condition (C) reduces to 
E[r'"+^] = ©((J''^^). Indeed, for any 7' > 1 we can write 

E[|t - (5r^2] < E[r''^2j < e[t'^+1]^ = 0(5'-+i)^Tf = 0(5'-v2-)^ 

where the first inequality is due to P(t > 0) = 1 and the second one follows 
from Holder's inequality and the fact that r V 2 < r + 1. Using this observation, 
we now specialize Theorem 2.2 to renewal processes. 

Corollary 2.1. Let r > 1 and suppose that (t„) is a renewal process, i.e., 
P(t > 0) = 1. // E[r''+i] = 0{S'-+^), then 



6Nt 
t 



1 



O 



and, consequently, 

SNt 
t 

When, in particular, qr\j2 = rW 2, 

SNt 

~r 

whereas when qry2 ~ 

SNt 



1 as t,S oo .such that S — o{t). 



- 1 



- 1 



= o 



Proof. When P(t > 0) = 1, it is I't = iVf + 1, thus, 



SNt 



< 



Sut 

T 



Applying (2.7) and taking expectations we obtain 



SNt 
t 



< Cr 



Svt 
t 



t 



Applying now Theorem 2.2 completes the proof. 



(2.16) 



(2.17) 
□ 
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Remarks: 

1. The condition of Corollary 2.1 is satisfied when t is positive and stochas- 
tically less variable than an exponential random variable cr with mean S, i.e., 
E[5(t)] < E[(7((t)] for every convex and increasing function g : (0,00) — > (0, 00). 
This is, for example, the case when r is new better than used in expectation, i.e., 
E[t - t\T >t]< E[t] V t (sec, e.g., [22], pp. 435-437). 

Indeed; if we denote by \r~\ the ceiling of r, i.e., \r~\ :— minjn e N : n > r}, 
then 



where the first inequality holds because x a;''+^ is a convex and increasing 
function on (0, 00), the second inequality follows from an application of Holder's 
inequality and the last step follows from the fact E[(t"] — 6" /nl for any n G N. 

2. Let (Yf ) be a spectrally negative Levy process, i.e., a stochastic process 
with stationary and independent increments and without positive jumps, so that 
E[Yt] = fit and E[(Y^~)''+-'^] < 00. Then, Corollary 2.1 also applies to the first 
hitting times of Y, i.e., when r = mi{t : Yt > A}, where A > and fi > 0. 

Indeed, from Wald's identity it follows that 6 = E[r] = A///, whereas from 
Theorem 4.2 in [15] we have 



Note that 5 goes to infinity if either A — > 00 or /i ^ 0. Thinking of (t,i) as 
sampling times of (Yt), it is natural to consider ^ as fixed and to assume that 
the sampling period S is controlled by threshold A. This implies that E[t''"'"-'^] = 
Q{S^'^^) as (5 -> 00 and proves that the condition of Corollary 2.1 is satisfied in 
this case as well. 

3. The previous remark still holds if y is a random walk, a case we consider 
in more detail in the next section. 

2.3. The variance for random walks with large drift 

Wc close this section establishing (1.4). 





] = (A//i)'-+i(l + o(l)) as A ^00. 



(2.18) 



Theorem 2.3. // Var[r] = Q{S^) and E[(r+)3] = 0{S^), then 



Var[T] t 



— >■ 1 as d,t 00 so that d = o(t). 
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Proof of Theorem 2.3. First of all, wc observe that 

5^ Var[t/t] = Var[(5z/t -t\= Var[Ji^t - t^, + r^, - t\ 

= Var[(5i/t - T^J + Var[r^j - t] + 2 Cov((5i/t - t^^ , r,,^ - t) 
= Var[T] Eh] + Var[r,, - t] + 2 E[(5i/t - r,J(T,, - t\ 

where the last equality follows from (2.3). Now, multiplying both sides by 
(5/(Var[r] t) we obtain 

j3Varh] _^Eh] <5Var[T.,-^] (5 Ep^/, - r.J(r., - t)] 
Var[r]t t Var[r] t Var[r] t ' ' 

From Theorem 2.1 it follows that (5E [i/t] =:t(l+o(l)), therefore it suffices to 
show that the second and third term in the right-hand side of (2.19) go to 0. 
Indeed, for the second term we observe that 

'^Varir^.-t] ^ >5 E[(r,, - t)^] ^ E[(r+)=^] ^ 0[^) ^ /^n 
Var[T]t - Var[r]i " ''^ Var[T] < e((52)t Vt/' 

where the second inequality follows from an application of (2.6) with r = 2 
and the first equality from the assumptions of the theorem. Regarding the third 
term in (2.19), from an application of the Cauchy-Schwartz inequality we have 

^{\bvt - t^Mt., - t)\ < VE[{5,yt~T,,y] E[(r,,-t)2] 



<Wc2MVarH E[(r+)3] 



5 

where the second inequality follows from (2.3) and (2.6). Consequently, 



SE[\Siy, -T^,\{T,,-t)] S Eh]Var[T] E[(r+)3] 

Var[T] t - Var[r] t V 5 



SEWt] E[(T+)3] 0{S^) _ ( 5 

where the second inequality follows from the assumptions of the theorem. This 
completes the proof. □ 

We now specialize Theorem 2.3 to the case of renewal processes. 

Corollary 2.2. Suppose that P(r > 0) = 1. // Var[T] = Q{5'^) and E[t^] = 
0{5^), then 

<53 VarfiVt] ^ , c / ^ 
!■ 1 as o,t — >■ c» so that o ~ o[t). 



Var 
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3. Low-rate parameter estimation 

Let (Xt)tgN be a sequence of i.i.d. random variables with non-lattice distribu- 
tion, mean := E[X], and variance :— Var[X] < oo, where X := Xi. From 
the Central Limit Theorem we know that the sample mean is an asymptotically 
normal estimator of /i, i.e., 

Vi{^lt - fJ.) ^ ^f (0,(7^) as t^oo, (3.1) 

where 5*4 := Xi + . . . + Xt and fit St/t. Our goal in this section is to construct 
estimators of fi that preserve this asymptotic distribution when the random walk 
(St) is only observed at a sequence of sampling times (t,, )„gN that form a renewal 
process with rate 1/(5. Therefore, in what follows, (r„ — r„_i)„gN is a sequence of 
positive, integer-valued, i.i.d. random variables with mean 6 := E[t] > 0, where 
T := Ti. Moreover, we will use the notation introduced in (2.1) and (2.2) and 
denote by Nt the number of acquired samples up to time t and hy vt ~ Nt + 1 
the first hitting time of level t. 

In this context, two natural modifications of fit are 

P-t Sr^jTNt, flt-.^Sr^Jt, teN. (3.2) 

That is, p,t is the sample mean evaluated at the most recent sampling time, 
whereas in /it we have replaced the most recent sampling time in the denomi- 
nator by the current time t. Note that (/It) coincides with {fit) at the sampling 
times, but takes smaller values in between. 

In order to describe the asymptotic behavior of both these estimators as 
5, < — > oo, we will need Lorden's upper bound on the expectation of the age, or 
undershoot, t — Tf^^: 

which is valid since P(t > 0) = 1 (see [17], page 526). Indeed, the following 
lemma, which will play an important role in the sequel, is a direct consequence 
of this bound. 

Lemma 3.1. // Var[T] ^ 0{5'^), then 

— ^ — ^■^ 1 as (5, t — > oo so that 5 = o{t). 

Proof. From (3.3) and the assumption of the lemma it follows that E[t — tatJ = 
0{5) as (5, t — > oo and, consequently, it is clear that 



1 

t \ t \t, 

which completes the proof. □ 
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3.1. Complete observations at arbitrary sampling times 

The following theorem provides sufficient conditions on the distribution of r so 
that Jit and jit have the same asymptotic distribution as /it when 5,t ^ oo. 

Theorem 3.1. //Var[r] = 0{5^), then 

(i) \/t(/2t — /i) — > A/'(0, (T^) as (5, >oo so that S ~ o{t), 

as (5, t — > oo so that S ~ o(Vt). 

Proof. From Lemma 3.1 it follows that r^^/t converges to 1 in probability as 
5, t — > oo so that 5 ~ o{t). As a result, from (3.1) and Anscombe's theorem (see, 
e.g., [16], page 17) we obtain (i). Then, in order to prove (ii), from Slutsky's 
theorem it follows that it suffices to show that 

Vt\flt~f^t\ = Vt\Sr^y—^ = \flt\^-^^ (3.4) 

t TNt Vt 

converges to in probability as (5, t — >■ oo so that S = o{Vt). Indeed, from the 
Strong Law of Large Numbers and Lemma 3.1 it follows that p,t converges to ^ 
in probability as (5, i — > oo so that 5 = o{t) (see, e.g.. Theorem 2.2 in [16], page 
12). Moreover, from (3.3) and the assumption of the Theorem it follows that 

^^LiIeA = nf—\ 

Vt yvt^' 

which implies that {t — Tiq^)/^/t converges to in probability as (5, t oo so 
that 5 = o{\/t) and completes the proof. □ 

Remarks: 

1. The proof of Theorem 3.1 remains valid when 5 is fixed as t — >■ oo. Thus, 
Theorem 3.1 suggests that whereas both estimators fit and jit are efficient for 
small or moderate 5, only /it will remain efficient for large sampling periods. 

2. When sampling at any given time t has probability p and is independent 
of the past, then r is geometrically distributed with mean 5 ^ \/p and variance 

Var[r] = i^= ,52(1 -5-1), 

thus, the condition of Theorem 3.1 is clearly satisfied in this case. 

3. When the random walk has positive drift and is sampled whenever it 
increases by a fixed amount A since the previous sampling instance, i.e.. 



r„ inf{t > T„„i : St - S'r„_i > A}, n e N, 



(3.5) 
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then r = inf{t : St > A} and 

(5 = ^ +0(1) and Var[r] = ^ a(i + o(l)) as A ^ oo, (3.6) 

(see, e.g., Theorem 9.1 in [16]). Therefore, assuming that the samphng period 
5 is controlled by A, we have Var[T] = Q{5) as (5 — ?> oo, which implies that the 
condition of Theorem 3.1 is satisfied in this case as well. 

4. When the sign of /i is not known in advance and the random walk is sam- 
pled whenever it either increases or decreases by A since the previous sampling 
instance, i.e., 

T„ :=inf{t>T„_i : |5t-5,„_J > A}, n e N, (3.7) 

then (3.6) still holds if ji is replaced by (see [18]) and, consequently, Theorem 
3.1 applies to this case as well. 



3.2. Censored observations 

We now focus on the case that the random walk has positive drift and the 
sampling times are given by (3.5), but the overshoots 

rin ■■= Sr„ - Sr„_, - l^, « G N, 

are unobserved. In other words, we do not learn by how much more than A has 
the random walk increased since the previous sampling instance. 
In this context, the estimators defined in (3.8), 

1 ^' r 1 1 ^* r 1 

A*™, = A + r7„ , /it = -V A + r;„, t e N, (3.8) 

Ti— 1 n— 1 

are no longer applicable. Our goal in the remainder of this section is to find 
estimators that rely only on the counting process {Nt) and still preserve the 
asymptotic distribution (3.1). 



3.2.1. Ignoring the overshoots 

If we simply ignore the unobserved overshoots, the estimators in (3.8) reduce to 
fLt:=^, fit:=^, t^n. (3.9) 

TNt t 

In order to describe their asymptotic behavior as 5, < — >■ cxj, we need the following 
lemma, which relies on the low-rate renewal theorem that we established in the 
previous section. Moreover, in what follows, we set rj := rji (similarly to t = ri 
and X = Xi). 
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Lemma 3.2. E[?/] ~ C>{\) as A — > oo and 
SNt 



t 



- 1 



d E[7V,] = (-)(l + o(l)) 



as 5, t —> oo so that 5 ~ oit) . 



Proof. The first claim of tlic lemma follows from the assumption that X has a 
finite second moment (see, e.g., Theorem 10.9 in [16]). Moreover, from (3.6) we 
have Var[T] = 0((S), thus, (72 = 1 and 02 = and the second claim of the lemma 
follows from Corollary 2.1. □ 

Theorem 3.2. (i) If 5, t —>■ 00 so that 5 ~ o{t), then 



(a) If S,t ^ 00 so that \ft << S << t, i.e., S = o{t) and \/t — o(S), then 

Vi(At -m) ^ A/'(0,^T^). 

Proof. From Wald's identity and the fact that (??ra)„gN is a sequence of i.i.d. 
random variables, it is clear that = A + E[?/], therefore 



iMt - Ml 



A a 

t ^ 



1 



SNt 
t 



(3.10) 



Taking expectations and applying Lemma 3.2 we obtain (i). 

From Theorem 3.1(i) we know that Vt{fLt — ^ A/'(0,ct^) as S,t ^ 00 so 
that 5 = o{t), thus, it suffices to show that 



Nt 



Nt 



v^lMt - Mil = — = — 4; X! ^" 

^ n—1 ^ ^ n—1 



converges to in probability as 5, i — >■ 00 so that \/t << S << t . Indeed, from 
Lemma 3.1 it follows that t/T^^ converges to 1 in probability as (5, i — >■ 00 so 
that S = o{t). On the other hand. 



where the inequality is due to i^t = A^t + 1, the first equality follows from an 
application of Wald's first identity and the second equality follows from Lemma 
3.2. □ 
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Remarks: 

1. The proof of Theorem 3.2 remains vahd, with obvious modifications, when 
the sign of /i is unknown and (3.5) is replaced by the two-sided samphng scheme 
(3.7). 

2. Theorem 3.2(ii) suggests that even if censoring is completely ignored, the 
inflicted performance loss can be negligible, as long as we use the last sampling 
time TjVt in the denominator, instead of i, and the sampling rate is sufficiently 
low, i.e., 5 » \/t (compare this with the case of complete observations in 
Theorem 3.1, where only an upper bound was imposed on the speed at which 
8 -)■ oo). 

3. Theorem 3.2(i) implies that fit is a (weakly) consistent estimator of \i as 
(5, i — > oo so that (5 = o(t). Moreover, since 




it is clear that the best sampling rate for [it is given \>y 8 — 0('\/t). However, 
even in this case, [it fails to preserve the asymptotic distribution (3.1) of /.Jt. 

4. Using similar ideas as in Section 4.2 of [12], it can be shown that ignoring 
the overshoots incurs no loss efficiency for the estimation of the drift of a Brown- 
ian motion, when the latter is sampled at discrete times but with high frequency. 
More specifically, in our notation, suppose that E[X] = [ih and Var[X] = cP'h, 
where /i > is the period at which the underlying Brownian motion is being 
sampled. Then it can be shown that Vt(jlt — m) ^ A/'(0,cr^) a.s S,t ^ oo and 
/i — )■ so that y/h = o{S/\/t)^ whereas this asymptotic distribution remains 
valid even if S is fixed. 

In the next subsection, we do not make this restrictive assumption and im- 
prove upon /It approximating the unobserved overshoots. 

3.2.2. Approximating the overshoots 

Assuming now that we know the distribution of the random walk, we construct 
modifications of fit and fit that rely on approximations of the unobserved over- 
shoots. More specifically, the main idea of the proposed approximation is to 
replace each rjn in (3.8) by its expectation, £[77]. However, since the latter is 
typically an intractable quantity, we use the limiting average overshoot instead. 
Indeed, it is well known from classical renewal theory (see, e.g., [24], page 171) 
that, since X is non-lattice and has a finite first moment, 77 converges in dis- 
tribution as A — > cxj to a random variable ^ whose density with respect to the 
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Lebesgue measure is 

PiSr^ > X) 

P(e e dx) := \ + ' dx, X > 0, 

where t+ := inf{t G N : 5t > 0} is the first ascending ladder time. Moreover, 
since X has a finite second moment, the limiting value of the expected overshoot 
£[77] as A — > 00 is finite and admits the following representation: 

p(m) lim EM - m = (3.11) 

We have expressed the limiting average overshoot as a function of /i in order to 
emphasize that it depends on the unknown parameter, /i, thus it cannot be used 
directly to approximate the unobserved overshoots. However, we can obtain a 
working approximation of if we replace /x with an estimator that does not 
require knowledge of the distribution of X, such as fit or fit- Doing so, we obtain 
the following estimators of ^: 



^ 2^ [A + p(Mt)J 5(Mt) 

(3.12) 



n=l 



^ n— 1 

where the real function g is defined as follows 

p{x) 



a; > 0, 



and the term in the parenthesis reflects the overshoot correction. We will show 
that, unlike fit, gift) preserves the asymptotic distribution (3.1) of fit, whereas 
g(/i() attains it for a wider range of sampling rates than fit- However, for this, 
we need to impose some additional conditions on the distribution of X that will 
guarantee that: 

(i) SNt/t converges to 1 as (5, i — >■ c» so that S = o{t) in C? 

(ii) the overshoot r\ is square integrable, i.e., £[77^] = 0(1) as A — )■ 00, 

(iii) the expected overshoot E[ri] converges sufficiently fast to its limiting value, 
in the sense that \E[t]] — p(/i)| = 0(1/A) as A — ?> 00. 

We will see that the first two requirements are both satisfied when the fol- 
lowing condition holds 

(Al) E[|X|'~+i] < 00 for some r > 2, 
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i.e., when the third moment of X is finite. For the third requirement, we will 
rely on the refinements of the renewal theorem due to Stone (see [25], [26]) and 
assume additionally that 



(A2) X is strongly non-lattice, i.e., liminf|| 
and 



|l-E[e 



>0, 



(A3) either X takes only positive values, i.e., P{X > 0) = 1, or it has an 
exponentially decaying right tail, i.e., P{X > x) = o(e~^i^) as x oo for 
some 9i> Q. 



Indeed, from [25] we know that if {A\) and {A2) hold, then 



C/+(A). 



p(m) 



(3.13) 



whereas from [26] it follows that if additionally X has an exponentially decaying 
right tail, then 



C/+(A) 



A p(/i) 



= 0(6"^^"^) as A ^ cx) 



(3.14) 



for some 82 > 0, where J7+(A) is the renewal function and (T„,+)„gN the se- 
quence of ascending ladder heights, i.e., 

00 

t/+(A) ^ ^".+ - ^ ^"-1.+ ■ ^* > ^--.-1.+ }- 



Lemma 3.3. (i) If E[{X+)^] < 00, then E[?/2] = 0{1) as A ^ 00. 
(ii) If E[{X^)^] < 00, then as (5, t —> 00 so that S ~ o{t) 



SNt 



O 



(5\2 



and E 



^■(^'(l + od)). 



(Hi) If{Al) - {A3) hold, then \E[ij] - p(^)| = o(A 



-(r-l) 



) as A ^ 00. 



Proof. For (i), we refer to Theorem 10.9 in [16]. From Theorem 8.1 in [16] it 
follows that if E[(J'('~)'^] < 00, then E[t^] — 0{6^), whereas from (3.6) we know 
that Var[r] = Q(S), thus (72 = 1 and a2 = 0. Therefore, from Corollary 2.1 we 
obtain (ii). For (iii), we first observe that 



\E[^]-pi^l)\ = \E[Sr^A]-pi^l)\^^l e[t] 



p(m) 



(3.15) 



where the second equality follows from an application of Wald's identity. When 
P{X > 0) = 1, then E[r] = U+{A), in which case (iii) follows from (3.13). 
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When X has an exponentially decaying right tail, using (3.14) and the following 
representation of the expected overshoot 



(1 - P{Sr+ > A + y-x)) U+idx), 



we can show (similarly to [7]) that |E[ry] — p(/i)| decays exponentially fast as 
A — > C!0. which implies the desired result. □ 

Theorem 3.3. // (Al) — {A3) hold and /i p{fi) is Lipschitz, then 

(i) Vi{g{fit) - m) ^ A/'(0,cr2) S^t^oo so that t^l'^ « 6 « Vi, 
(a) \/t{g{jlt) - m) ^ -^(0, cr^) as 5,t ^ oo so that i^/"* «5 «t. 

Proof, (i) From Theorem 3.1(ii) we know that Vt{fLt — /i) ^ A/'(0, cr^) as (5, t — >■ 
oo so that 6 = o{\/t). Therefore, due to Slutsky's theorem, it suffices to show that 
Vi I/it ~ g{jj.t)\ converges to in probability as 5, t oo so that i^/** << 5 << t. 
In particular, since 



1 I / 

Vi\fit - 9ih) \ = -y= l^^yin - pifj-t) 

^ n—1 



^ n—1 



Nt 



Nt\EM-pip)_[^Nt\p{pt)-p{p)\ 



Vi 



(3.16) 



it suffices to show that each term in (3.16) converges to in probability when 
t^^^ « 5 « t. We start with the first term and we observe that, since vt = 
Nt + 1, using the triangle inequality we can write 



Nt 



j2Ut-m) <\J2['in-m 



EW|. 



Taking expectations and applying the Cauchy-Schwartz inequality to both terms 
of the right-hand side we obtain 



I't 



iv.t - E[^ 



< 



•^t 

Var[^(,„-E|,,] 



\ 



VEhJVarM + v/Eh]Var[77], 



e[£(77„-EM 



G. Fellouris/ Low-rate renewal theory and estimation 



20 



where the second inequahty is trivial and the cquahty fohows from an appUcation 
of Wald's identities. Consequently, from Lemmas 3.2 and 3.3 we obtain: 



^E[|i:(,„-EM 



< 2 



Var[77] /<SEh] 



t 



O 



(3.17) 



Regarding the second term in (3.16), from Lemma 3.3(iii) and the fact that 
5 = 9(A) it follows that |E[?7] — p{^) \ — o{\/5) as (5 — > oo. From this observation 
and Lemma 3.2 we obtain: 



t 



(3.18) 



Regarding the last term in (3.16), from the assumption that ji — > p{ii) is 
Lipschitz, there is a constant C > so that 



Nt \p{jit) - p{^i)\ < c ^* ~ ^' < c ^* 



SNt 
t 



1 



-CE[r,] 



where the second inequality follows from (3.10). Then, taking expectations and 
applying the Cauchy-Schwartz inequality in the first term we obtain 



Nt\p{iit)- p{fj-)\ 



- 1 



c 



mi 

t3/2 



o 



^2 ) 



= o 



6"^ J' 



(3.19) 



where the first equality follows from Lemma 3.3(ii). 
From (3.16), (3.17), (3.18) and (3.19) we obtain 



Vt 



Pt - 9[ 



1 ^* 
(At)l = -q^^iJ^^^n- pip. 



71 = 1 



O 



U2 



(3.20) 



which completes the proof of (i). 

(ii) According to Theorem 2.2, Vt{p,t — p) ~^ A/'(0, cr^) as (5, i — >■ 00 so that 
5 = o{t), therefore it suffices to show that \/t\p,t — 5 (At) I converges to in 
probability as (5, i — > 00 so that t^^^ « S « t. Since 



Pt - 9{ 



Vt 



Nt 



< 



t 
TNt 



Nt 



TNt Vt 



"yf I E(^" " ^(^0 



n=l 

Nt\p{pt)- p{ilt)\ 

Vt 



(3.21) 
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from Lemma 3.1 and (3.20) it is clear that it suffices to show that the second 
term in (3.21) converges to in probability as J, t — > oo so that 5 = o{t). Indeed, 
due to the assumption that fi — > p{pL) is Lipschitz, we have 



Nt\piftt)-piftt)\ ^^Nt 



ANf ANf 



TNt 



t 



C 



From Lemma 3.1 and Lemma 3.3(ii) it follows that the right-hand side converges 
to in probability as (5, t — > oo so that S ~ o{t), which completes the proof. □ 

Remarks: 



1. When P{X > 0) = 1 and = cp^ for some c > 0, then p(/i) is a linear 
function of p. Indeed, from (3.11) it follows that 



1 + c 



2p 



■/i. 



For example, if X follows the Gamma distribution with shape parameter fc > 
and rate parameter A > 0, then = /x^/fc and p(/x) is proportional to /x for any 
given fc. 

2. When P(X > 0) < 1, p(/i) does not typically admit a convenient closed- 
form expression in terms of /x. An exception is the Gaussian distribution, in 
which case (3.11) takes the following form 



2p 



K) - 6„$(-6„) 



a 



(see [30], page 34), where $(•) and 0(-) arc the c.d.f. and p.d.f. respectively of 
the standard normal distribution. In this case as well, if = cp? for some 
c > 0, then 6„ = \pnjc and consequently p{\i) = Wc fJ-, where 



1 + c 



E 

n=l 



\fnfc 



3.3. Summary and simulation experiments 

In Table 1, we present the estimators of p that we considered in the context of 
the sampling scheme (3.5). For each estimator, we report whether it needs the 
overshoots (?7„) to be observed or not, whether it uses the sampling times (r„) 
or only the counting process {Nt), whether it assumes knowledge of the distri- 
bution of X . Moreover, we specify the sampling rates at which its performance 
is optimized. 
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In Figure 3.3 wc plot the relative efficiency of each estimator as a function 
of the sampling period, (S, for a fixed horizon of observations, t ~ 300, when 
X ^ M{iJ,, cfi^) and c = /i = 4. More specifically, we define the relative efficiency 
of fit (similarly for the other estimators) as 



and we compute it using simulation experiments. This figure offers some very 
interesting observations which are in complete accordance with our asymptotic 
results. 

First of all, we observe that fit, fit and g{(j-t) are more efficient than jit, fit and 
g{pt) respectively for any S and especially for large values of 6. Thus, it is always 
preferable to use rjVj in the denominator, instead of t, especially when one is 
interested in large sampling periods, irrespectively of whether the overshoots 
are observed or not. 

Moreover, we see that for fit and jit, the smaller the sampling rate, the better 
the performance; for fit and g{p-t), performance deteriorates for small 5 and 
remains flat (and quite high) as S increases; for jit and g{fit), performance is 
optimized when (5 is in a particular range and deteriorates for both very small 
and very large values of S. Therefore, when the random walk is fully observed at 
the sampling times, it is preferable to have a high sampling rate, whereas when 
there is censoring and ryvj (resp. t) is used in the denominator, the sampling 
rate should be low (resp. somewhere in the middle). 

Finally, we observe that g{fj.t) is more efficient than fit uniformly over 6, 
whereas its performance attains its optimum over a much wider range for 6. On 
the other hand, g{fit) is more efficient than fit only for small 6. For very large 
values of S, fit turns out to be more efficient even than fit - Therefore, when using 
TNt (resp. t) in the denominator, approximating the overshoots is beneficial for 
high sampling rates (resp. for any sampling rate and especially for very low 
sampling rates). 



4. Conclusions 



In the present work, we extended some fundamental renewal-theoretic results 
for renewal processes whose rates go to and, more generally, for random walk 
whose drifts go to infinity. These extensions can be useful in a variety of setups 
in which recurrent events do not occur frequently. In the present work, they 
were motivated by the estimation of the drift of a random walk that is sampled 




Fig 1 . Relative efficiency against sampling period for each of the estimators in Table 1 in a 
curved Gaussian distribution. The horizon of observations is t = 300, the true mean is fi = A 
and the variance cr-^ = . The computations are based on simulation experiments. 



Estimator 


Formula 


Distribution 


Overshoots 


Sampling Times 


Optimal Rate 






no 


yes 


yes 


<5 « t 


h 




no 


yes 


no 


s «Vt 


fit 




no 


no 


yes 


5 « t 


fit 




no 


no 


no 


5^ Vt. 


aifi't) 




yes 


no 


yes 


i^/^ «S«t 


g(fi-t) 




yes 


no 


no 


«s«Vt 



Table 1 

Estimators of fj, for sampling times given by (3.5). The first two rows also apply to more 
general sampling schemes. For each estimator, we report whether it requires knowledge of 
the random walk distribution, the overshoots {rjn), the sampling times (t„) and we specify 
the sampling rate for which its performance is optimized. 
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at a sequence of random times so that its increments are censored. We consid- 
ered different estimators, which either ignore or approximate the unobserved 
overshoots, and we showed that they attain the asymptotic distribution of the 
sample mean for appropriate, asymptotically low sampling rates, under certain 
conditions on the random walk distribution. Our asymptotic results were sup- 
ported by simulation experiments, which showed that it is always preferable 
to use the most recent sampling instance, instead of the current time, in the 
denominator of the estimator and that approximating, instead of ignoring, the 
unobserved overshoots can be very helpful when one wants to sample the process 
at a faster rate. 
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